Interactive-Time Similarity Search for Large Image Collections Using Parallel VA-Files

Nearest-neighbor search (NN-search) plays a key role for content-based retrieval. As a first contribution, this article shows that NN-search is a meaningful implementation of similarity search, even if features are high-dimensional. But NN-search over high-dimensional features is of linear complexity and query response times have not been satisfactory for large collections of multimedia objects. This paper, based on the VA-File, investigates parallel NN-search in a Network of Workstations (NOW). The article identifies various design alternatives for such a search engine and evaluates them. The alternatives basically relate to data placement and division of work among components. We also use Amdahl’s law to predict the speedup and response times for a given data set and a given setup. Because of the scan-based nature of the VA-File, one might expect an improvement almost linear in the number of components. But the best speedup we have observed is by almost 30 for a NOW with only three components. The effect is due to the elimination of the IO-bottleneck. From another perspective, our solution provides interactive-time similarity search, i.e. a search through 900 MB feature data lasts about one second in a NOW with three components.